Almost Realtime Live Data Visualization in QGIS (Air Traffic Use Case)

When working in a GIS software like QGIS, mostly we are working with static data like street, building, land cover, etc. Or might be a data which has time information so we can visualize the temporal change. What about visualize live data in almost real time? Do we need a GIS server, cloud or map service? I think this is an interesting topic, and I will discuss about it in this post with live air traffic data use case.

In the previous post, I made a tutorial how to build a flight tracking application with open air traffic data in Python. The application is running in a web browser and the flight data will be updated in a specified time interval. In this tutorial we will do the same thing in QGIS. We will visualize the air traffic live data on QGIS map canvas and get the update data in every five or ten seconds. At the end of this tutorial we will get an almost realtime airplanes' location within an area as in figure 1 below.
 
Air Traffic Live Data in QGIS
Figure 1. Air Traffic Live Data in QGIS. Airplanes are queueing for landing at San Fransisco International Airport
 
Figure 2 is the schema that shows how the system works. Can be seen from the schema that Python plays role for requesting the data, get the response, process it and store the data into a CSV plain text. On the other side, QGIS will render the airplanes position based on the data and refresh it with in an interval to get the latest information. To make it happen, we don't need any GIS server, cloud or map service. Anyone could do this as long as there is internet connection available with Python and QGIS installed in a machine.

Schema how the system works (QGIS Live Data)
 Figure2 . Schema how the system works

Based on the schema, this tutorial consist of several sub-topics, such as: getting the data (send the request and process the response), Plotting the data on QGIS map canvas and render it within a time interval. Let's get started!

Getting Air Traffic Live Data

The data for this tutorial is coming from OpenSky Network which is an association that provides air traffic data around the globe. There are some APIs that can be used to retrieve data from OpenskyNetwork such as: Python API, Java API and REST API. In this tutorial we will use REST API to retrieve data within a specified boundary area.

To retrieve air traffic data within an area we need to define minimum and maximum coordinate in geographic coordinate system. For example I want to fetch all planes over United States with minimum and maximum coordinate respectively -125.974,30.038 and -68.748,52.214. The REST API query to request the data anonymously will be as follow:

https://opensky-network.org/api/states/all?lamin=30.038&lomin=-125.974&
lamax=52.214&lomax=-68.748

The anonymous request has resolution 10 seconds, it means we can send the request in every 10 seconds. On the other hand if you are a registered user, the resolution will be faster, about 5 seconds. To make a request as registered user, the username and password must be include in the query. Then the query will be:

https://username:password@opensky-network.org/api/states/all?lamin=30.038lomin=-125.974& lamax=52.214&lomax=-68.748

To try the query, simply copy it and paste into a browser. If you get a response like figure 3, means it works and we are ready to continue to the next step. Furthermore if you want to know in more detail about air traffic data response from OpenSky Network please visit REST API Documentation.

Live Air Traffic Data Response
Figure 3. Live Air Traffic Data Response

Sending Request and Process The Live Data Response

In this step we will write Python code to request the live air traffic data and process the response. The complete code can be found at the end of this section.

We are starting with importing some libraries namely: requests, json, csv and time. Then define the coordinate extent with minimum and maximum coordinate. Next at line 16 an output path where the response data will be stored is specified, so make sure to change with yours. If you are a registered OpenSky Network user, giver your username and also the password in user_name and password variable at line 19-20. The last part of the code is used to send the query using requests, get response in JSON format and save it into a CSV file. This process will be done in a loop within interval 10 seconds for anonymous request or 5 seconds for a registered user.  

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
'''
LIVE AIR DATA TRAFFIC REQUEST
by ideagora geomatics | www.geodose.com | @ideageo
'''
#IMPORTING LIBRARIES
import requests
import json
import csv
import time

#AREA EXTENT COORDINATE GCS WGS84
lon_min,lat_min=-125.974,30.038
lon_max,lat_max=-68.748,52.214

#CSV OUPUT PATH
csv_data='/home/data.csv'

#REST API QUERY
user_name=''
password=''
url_data='https://'+user_name+':'+password+'@opensky-network.org/api/states/all?'+'lamin='+str(lat_min)+'&lomin='+str(lon_min)+'&lamax='+str(lat_max)+'&lomax='+str(lon_max)
col_name=['icao24','callsign','origin_country','time_position','last_contact','long','lat','baro_altitude','on_ground','velocity',       
'true_track','vertical_rate','sensors','geo_altitude','squawk','spi','position_source']

#REQUEST INTERVAL
if user_name !='' and password !='':
    sleep_time=5
else:
    sleep_time=10

#GET DATA AND STORE INTO CSV
while col_name !='':
    with open(csv_data,'w') as csv_file:
        csv_writer=csv.writer(csv_file,delimiter=',',quotechar='"',quoting=csv.QUOTE_ALL)
        csv_writer.writerow(col_name)
        response=requests.get(url_data).json()
        
        try:
            n_response=len(response['states'])
        except Exception:
            pass
        else:
            for i in range(n_response):
                info=response['states'][i]
                csv_writer.writerow(info)
    time.sleep(sleep_time)
    print('Get',len(response['states']),'data')

Save the code with Python extension (.py) and run it from a command prompt or terminal. Type python or python3 if you use python 3 followed by the file name. The code will be running as in figure 4 below. 

Don't close the terminal, because it will work continuously to get the latest air traffic data from OpenSky Network, and we will use it in QGIS.

Flight Data Request
Figure 4. Flight Data Request

Visualize Live Air Traffic Data in QGIS

We already get the live data streaming, now let's visualize it in QGIS with the following steps.

Add the CSV data into QGIS. From Data Source Manager, select Delimited Text in the left menu. Then in the right side, select  the File Name. In Geometry Definition section select long column for X field and lat column for Y field. Make sure to get the Sample Data correctly. If not try to change the delimiter properties in File Format section with Custom delimiters option.     

Add flight data QGIS
Figure 5. Add Air Traffic Data to QGIS

After pushing the Add button in the Data Source Manager, the air plane's position within the requested area will be plotted on QGIS map canvas. To make it more meaningful in a geospatial extent, add a basemap. To add a basemap I used Tile+ plugin which provides some popular basemaps. For this case I used STAMEN TERRAIN basemap.  Figure 6 shows all aircraft's position over the US with STAMEN TERRAIN basemap.

Aircraft Position in QGIS Map Canvas
Figure 6. Aircraft Position in QGIS Map Canvas

So far we already get airplane position in a static way. The position of airplanes will not change because it doesn't fetch any updated data. Therefore in this last step, we will make it dynamic. The position of aircraft will be updated every 5 or 10 seconds. Then we will change the dot marker with airplane icon and also rotate it respectively with the track direction.

Firstly let's change the dot marker into airplane icon. Right click on data layer and then select Properties. On the left menu select Symbology and chose topo airport icon as in figure 7.

Change Symbology
Figure 7. Change Symbology

To set rotation angle of the marker, select the menu at the right of Rotation parameter then  select Field type:.... and then select true_track column (see figure 8). 

Set Rotation Angle
Figure 8. Set Rotation Angle
 

Before proceeding to the next step, click Apply or OK button. You should see the airplane marker and  it rotates in flight direction angle as shown in figure 9.

Airplane marker with it's rotation angle
Figure 9. Airplane marker with it's rotation angle

Finally let's update the data within a time interval. Select data layer and choose Properties again. On the left menu select Rendering. In the right window check Refresh layer at interval (seconds) option, and set it to 5 or 10 as in figure 10. It means the layer will refreshed every 5 or 10 seconds. If there is any data change after refreshing is taking place, the position of airplanes will be updated and we get an almost realtime air traffic live data that visualized in QGIS as seen before in figure 1.

Set refresh layer interval time
Figure 10. Set refresh layer interval time

That's all this tutorial how to visualize an almost realtime live data in QGIS. I think  this approach can be applied for other cases like visualize data from a sensor that taking measurement in a field like water level, temperature, humidity, and many more. Hope this post could inspire you and thanks for reading!

In another post I wrote a tutorial how to stream air traffic live data to Youtube. Check out this tutorial if you're interested.


Related Posts

Disqus Comments